Click any tag below to further narrow down your results
Links
MotherDuck's Jev is a decision model that classifies text by picking from predefined answers, running 30x faster than GPT at 1% of frontier model costs. The article walks through five real use cases—grading AI agent logs, analyzing Hacker News sentiment, screening sales calls, and others—with actual SQL queries and datasets you can attach and rerun.
- Jev classifies 100,000 items in 82 seconds at ~1% the cost of frontier models by constraining outputs to a closed list you define, returning typed columns with confidence scores.
- On Hacker News mentions, 89% of pre-2023 "Claude" and "Anthropic" matches were false positives (Claude Shannon, the anthropic principle), but Jev's name-checking reduced false positives to 0.8% post-2023 when the products actually launched.
- The pattern separates facts from judgment: SQL extracts exact counts from logs or data, Jev only answers subjective questions like "was this turn efficient?" or "is this negative?", keeping costs down while maintaining accuracy.
Pangram Labs released Pangram 4, an AI text classifier that detects whether text was written by humans or AI systems. The model achieves 99.16% accuracy and can now identify mixed human-AI writing and fine-grained edits better than its predecessor.
- AUROC of 0.9916 with 0.0041% false positive rate and 0.3396% false negative rate
- Can distinguish fine-grained edits and detect interleaved AI assistance in co-authored text
- Shows improved robustness to adversarial attacks and better generalization to out-of-distribution data
This article explores an unconventional method for classifying text by leveraging compression algorithms. The author demonstrates how to concatenate labeled documents, compress them, and use the compressed sizes to predict labels for new texts. While the method shows promise, it is computationally expensive and generally underperforms compared to traditional classifiers.
- Compression-based classification (concatenate labeled texts, measure compressed-size increase) hits a 0.749 macro F1 with gzip on 4 categories of 20 Newsgroups, versus 0.88 for multinomial Naive Bayes.
- lzma pushes accuracy up to 0.897, beating Naive Bayes, but takes 32 minutes versus over 5 minutes for gzip on just 1,353 test cases—far too slow to be practical.
- The technique reframes text classification as an information-theory problem, showing compression algorithms implicitly model word probability distributions.